1. Field of the Invention
The present invention relates to an image processing apparatus or an object inspecting apparatus which obtains information relating to an object by processing image data representing an image of the object taken by an image taking device, and to a recording medium on which an image-processing or object-inspecting control program is recorded.
2. Related Art Statement
An example of the above-indicated image processing apparatus is disclosed in U.S. Pat. No. 5,754,677. The prior image processing apparatus measures, by using a seek template and a measure template, a position, a rotation angle, and at-least one dimension of an object. The position of the object being image-processed is expressed in terms of the coordinates of a specific point of the object, such as the center point of the object, or an arbitrarily designated point of the object. The rotation angle of the object is expressed in terms of the angle of rotation of the object from its reference rotation position. The seek template includes a plurality of pairs of points. It is a general rule that when one of the two points of each pair out of the pairs is located inside an outline of the object and the other point of the each pair is located outside the outline, the processing apparatus judges that the object being image-processed is an object being sought by the seek template. To this end, the pairs of points defining the seek template are determined based on the shape of the outline of the object. However, in the case where it is known in advance that the position and/or rotation angle of the object may greatly vary, the processing apparatus repeatedly judges whether the object is an object being sought, while changing the position and/or rotation angle of the seek template. In this case, the processing apparatus produces a master seek template based on the outline of a reference object which has one or more reference dimensions and which takes reference position and rotation angle. Then, the processing apparatus produces various individual seek templates by changing the position and/or rotation angle of the master seek template, and virtually superposes each of the individual seek templates on a screen on which the image of the object is present. If any one of the individual seek templates satisfies the above-indicated rule, the processing apparatus judges that the object being image-processed is an object being sought. The object being image-processed or the object being sought may be a single, independent article as a whole, a portion of an article, or a projection, a hole, or a character present on an outer surface of an article. In the case where an object being sought has a simple shape, such as a quadrangle or a circle, a seek template including a small number of pairs of points is used. On the other hand, in the case where an object having a complex shape is sought, it is needed to use a seek template including a great number of pairs of points.
If the object being image-processed is judged as an object being sought, then the image processing apparatus measures, by using a measure template, the position, rotation angle, and/or dimensions of the object. In this connection, the object being image-processed that is judged as an object being sought is deemed as an object being measured. The image processing apparatus automatically produces the measure template based on the seek template with which the object being image-processed is judged as an object being sought. The simplest measure template is defined by a plurality of straight segments each of which connects between the two points of a corresponding one pair out of the pairs of points defining the seek template. As far as the present application is concerned, those straight segments are referred to as the xe2x80x9csearch linesxe2x80x9d. The processing apparatus determines respective points at which the search lines intersect the outline of the object being measured. In the case where an optical characteristic (e.g., degree of lightness) of an image of the object differs from that of an image of the background of the object, the luminance of the two images significantly changes at the positions of the outline of the object. The positions where the luminance significantly changes will be referred as the xe2x80x9cedgexe2x80x9d of the object, and respective points where the search lines intersect the edge will be referred to as the xe2x80x9cedge pointsxe2x80x9d. That the luminance significantly changes at the edge of an object being measured means that the rate of change of the luminance is greater than a predetermined value at the edge. In many cases, the outline of an object being measured is determined as the edge of the object. In those cases, the image processing apparatus can measure the position, rotation angle, and/or dimensions of the object, by using the measure template, i.e., by obtaining the set of the edge points present on the outline of the object. In the case where an object being measured has a simple shape, such as a quadrangle or a circle, a measure template including a small number of search lines is used. On the other hand, in the case where an object having a complex shape is measured, it is needed to use a measure template including a great number of search lines. In addition, in many cases, the number of search lines included in a measure template is greater than the number of pairs of points included in a seek template.
However, an image of an object may have an edge at positions where an optical characteristic (e.g., color) of the image other than degree of lightness significantly changes. In the above description, it is assumed for easier understanding purposes only that the two points of each pair out of the pairs of points of a seek template are located inside and outside the outline of the object, respectively, and the search lines of a measure template intersect the outline of the object. In fact, the search lines have only to intersect the edge of the object, and the two points of each pair have only to be located on both sides of the edge, respectively. In the above description, it is also assumed for easier understanding purposes only that the image processing apparatus performs, one time, a measurement using a measure template. However, in many cases, the processing apparatus performs two or more measurements using respective measure templates. In those cases, after the processing apparatus performs one measurement using one measure template, the apparatus additionally produces a new measure template including a greater number of search lines, or a new measure template having a smaller positional error with respect to the object being measured, and performs a more accurate measurement using the new measure template.
As is apparent from the foregoing description, the image processing apparatus can quickly seek, using a seek template, an object and quickly measure, using a measure template, a position, a rotation angle, and/or one or more dimensions of the object, even if the position and/or rotation angle of the object may be greatly deviated from its expected position and rotation angle.
It is an object of the present invention to provide an image processing apparatus or an object inspecting apparatus which further improves the above-described image processing apparatus.
It is another object of the present invention to provide a recording medium on which an image-processing control program or an object-inspecting control program is recorded.
The present invention provides an image processing apparatus, an object inspecting apparatus, and a recording medium which have one or more of the technical features that are described below in respective parenthesized sequential numbers (1) to (19). Any technical feature which includes another technical feature shall do so by referring, at the beginning, to the parenthesized sequential number given to that technical feature. Thus, two or more of the following features may be combined, if appropriate. Each technical feature may be accompanied by a supplemental explanation, as needed. However, the following technical features and the combinations thereof are just examples to which the present invention is by no means limited.
(1) According to a first feature of the present invention, there is provided an image processing apparatus, comprising an image taking device which takes an image of an object; an image-data memory in which image data representing the taken image of the object is stored; negative-search-line producing means for producing at least one negative search line which is expected not to intersect an edge of the image of the object represented by the image data, an optical characteristic of the image significantly changing at the edge; and judging means for judging whether the at least one negative search line intersects the edge of the image of the object. That the optical characteristic of the image significantly changes at the edge means, for example, that the rate of change of the optical characteristic of the image is greater than a reference value at the edge. The previously-described prior image processing apparatus uses a measure template including xe2x80x9cpositivexe2x80x9d search lines which are expected to intersect an edge of an image. Since the present image processing apparatus uses one or more xe2x80x9cnegativexe2x80x9d search lines which are expected not to intersect any edges of an image, it can do what could not have been done by the prior apparatus. The negative search lines may be used for inspecting a component or a part for finding one or more possible defects such as a scar occurring to its outer surface, or dirt adhered to the outer surface. For example, in the case where a component is inspected for finding a scar possibly occurring to its horizontal, rectangular outer surface, a number of negative search lines are superposed on an area inside the outline of image of the outer surface and it is judged whether at least one of the search lines intersects at least one edge of the image. For example, if a dark image of a scar or dirt is present in a light image of the outer surface, the outline of the scar or the dirt provides an edge of the images. Since the negative search lines are located inside the outline of image of the outer surface, they are expected not to intersect any edges of the image. However, if there is a scar or dirt on the outer surface, the negative search lines may intersect the image of the scar or the dirt and accordingly may intersect the edge of the image. Therefore, it is possible to judge whether there is a scar or dirt on an outer surface being inspected, by judging whether there is, on each of the negative search lines, an edge point where, e.g., luminance significantly changes. In order to distinguish search lines used for inspecting an object for finding one or more possible defects, from those used for measuring a position, a rotation position, and/or at least one dimension of an object, the former lines will be referred to as inspecting search lines and the latter lines will be referred to as measuring search lines. Negative search lines can be used as not only inspecting search lines but also seeking search lines for seeking an object as will be explained in detailed description of preferred embodiments. In addition, positive search lines can be used as not only measuring search lines but also seeking search lines for seeking an object and inspecting search lines for finding a defect of an object. In the last case, the judging means may judge whether one or more positive search lines intersect an edge of an image of an object and, based on the positive or negative judgment, judge whether an object being image-processed is an object being sought, and/or judge whether an object being image-processed has a defect.
(2) According to a second feature of the present invention that includes the first feature (1), the image processing apparatus further comprises a measuring device which measures at least one of a position, a rotation angle, and a dimension of the object, and the negative-search-line producing means produces the at least one negative search line based on the measured one of the position, the rotation angle, and the dimension of the object. As described above, the negative search lines are expected not to intersect the edge of the image. Accordingly, in the case where the negative search lines are used for inspecting an object for finding its scar or dirt, it is usually needed to know a position, a rotation angle, and/or at least one dimension of the object. In a particular case where an object being inspected has accurate known dimensions and is accurately positioned at known position and rotation angle, the measuring device may be omitted. However, in the case where an object (being image-processed before being judged as an object being sought) has inaccurate dimensions or is not accurately positioned at known position and rotation angle, it is needed to measure an inaccurate one or ones of the dimensions, position, and rotation angle of the object, before the negative search lines are produced. Thus, the image processing apparatus including the measuring device can inspect an object for finding its possible defect, even if the object may not have accurate dimensions or may not be accurately positioned at known position and rotation angle.
(3) According to a third feature of the present invention that includes the first or second feature (1) or (2), the image processing apparatus further comprises positive-search-line producing means for producing at least one positive search line which is expected to intersect the edge of the image of the object represented by the image data. Since the present image processing apparatus can use both the negative and positive search lines, the apparatus can seek or inspect an object with improved degree of freedom, reliability, and/or efficiency, as compared with the case where either the negative or positive search lines are used.
(4) According to a fourth feature of the present invention that includes the third feature (3), the judging means judges, when the at least one negative search line does not intersect the edge of the image of the object and simultaneously when the at least one positive search line intersects the edge of the image of the object, that the object is an expected object. The present image processing apparatus can seek an object with still improved reliability and efficiency. For example, in the case where a plurality of identical objects are arranged in an array, the present apparatus can quickly seek a particular one of the identical objects. Providing that three identical circular objects are arranged in a horizontal array at a regular interval, the present apparatus can easily seek, for example, the right-hand-side end one of the three circular objects, by first producing a vertical positive search line and a vertical negative search line distant rightward from the positive search line by the same distance as the regular interval, then moving the two search lines relative to the three circular objects, and finally identifying, as the right-hand-side end circular object being sought, one of the three circular objects whose edge is intersected by the positive search line and is not intersected by the negative search line.
(5) According to a fifth feature of the present invention, there is provided a recording medium on which an image-processing control program is recorded which is readable by a computer and is usable by the computer to process image data representing an image of an object taken by an image taking device, the image data being stored in an image-data memory, the program comprising the steps of producing at least one negative search line which is expected not to intersect an edge of the image of the object represented by the image data, an optical characteristic of the image significantly changing at the edge, and judging whether the at least one negative search line intersects the edge of the image of the object. The image processing apparatus according to the first feature (1) may be provided by the combination of the present recording medium and a computer.
(6) According to a sixth feature of the present invention, there is provided an object inspecting apparatus, comprising an image taking device which takes an image of an object; an image-data memory in which image data representing the taken image of the object is stored; search-line producing means for producing at least one of (a) at least one negative search line which is expected not to intersect an edge of the image of the object represented by the image data, an optical characteristic of the image significantly changing at the edge, and (b) at least one positive search line which is expected to intersect the edge of the image; and inspection judging means for judging that the object passes the inspection, when a condition is satisfied that when the at least one negative search line is produced, the at least one negative search line does not intersect the edge of the image of the object and/or, when the at least one positive search line is produced, the at least one positive search line intersects the edge of the image of the object, and judging that the object does not pass the inspection, when the condition is not satisfied. The previously-described prior image processing apparatus uses search lines exclusively for measuring an object being image-processed. In contrast, the present object inspecting apparatus uses search lines for inspecting an object being image-processed. Thus, the claimed invention widens the width or range of use of the search lines.
(7) According to a seventh feature of the present invention that includes the sixth feature (6), the object inspecting apparatus further comprises a measuring device which measures at least one of a position, a rotation angle, and a dimension of the object, and the search-line producing means produces the at least one of (a) the at least one negative search line and (b) the at least one positive search line, based on the measured one of the position, the rotation angle, and the dimension of the object. The present object inspecting apparatus can inspect, without any problems, an object which may not have a constant dimensions or may not be positioned at constant position and rotation angle.
(8) According to an eighth feature of the present invention that includes the sixth or seventh feature (6) or (7), the at least one of (a) the at least one negative search line and (b) the at least one positive search line comprises at least one finite line having opposite ends. In a particular case, a negative or positive search line may be provided by a line having at least one infinite end (in fact, the infinite end is limited or cut by the boundaries or outer frame of the image-process area). However, in many cases, the negative and positive search lines are provided by finite lines each having opposite ends. Each of the eighth to fifteenth features may be combined with each of the image processing apparatus according to the first feature (1), the object inspecting apparatus according to the seventeenth feature (17), the recording medium according to the fifth feature (5), and the recording medium according to the sixteenth feature (16).
(9) According to a ninth feature of the present invention that includes the eighth feature (8), the at least one finite line comprises at least one straight segment. Negative and/or positive search lines provided by straight segments are the simplest and are very useful.
(10) According to a tenth feature of the present invention that includes any one of the sixth to ninth features (6) to (9), the at least one of (a) the at least one negative search line and (b) the at least one positive search line comprises at least one closed line. In the case where the present apparatus inspects an object whose image is taken as having a circular outline, it is preferable to employ circular negative or positive search lines.
(11) According to an eleventh feature of the present invention that includes any one of the sixth to tenth features (6) to (10), the image taking device comprises a plurality of image-take elements each of which produces an electric signal representing an optical characteristic value of a corresponding one of a plurality of picture elements of the image of the object, the image-data memory stores, as the image data, the respective optical characteristic values represented by the respective electric signals produced by the image-take elements, such that each of the respective optical characteristic values is associated with a position of a corresponding one of the picture elements in the image of the object, and the apparatus further comprises a virtual-point designator which designates an arbitrary virtual point on a virtual screen corresponding to a physical screen defined by the image data stored in the image-data memory; and virtual-point-data calculating means for calculating an optical characteristic value of the designated virtual point based on the respective optical characteristic values of at least two picture elements of the image that correspond to the designated virtual point, and the search-line producing means produces, on the virtual screen, the at least one of (a) the at least one negative search line and (b) the at least one positive search line. In the case where the image taking device of the object inspecting apparatus that includes any one of the sixth to tenth features (6) to (10) takes an image of an object, an image-take surface of the image taking device may be divided into many picture elements, and many sets of picture-element data corresponding to the picture elements may be produced as the image data and stored in the image-data memory. However, the phrase xe2x80x9cthe image of, the object represented by the image dataxe2x80x9d used in the first feature (1) just means the image of the object obtained based on the image data. Accordingly, the image may be either a xe2x80x9cdiscrete-typexe2x80x9d image which is defined by a matrix of picture elements having respective optical characteristic values, or a xe2x80x9ccontinuous-typexe2x80x9d image which is not defined by any picture elements but whose arbitrary point has an optical characteristic value. The xe2x80x9cdiscrete-typexe2x80x9d image may be an image taken by the image taking device which includes a matrix of image-take elements each of which produces an electric signal representing an optical characteristic value of a corresponding one of a matrix of picture elements of the image. The discrete-type image can be said as a physical image because it is physically formed on the image-take surface defined by the matrix of image-take elements. The screen on which the physical image is present is referred to as the xe2x80x9cphysicalxe2x80x9d screen. The physical screen coincides with the image-take surface. Image data representing the physical image may be stored in the image-data memory, such that each of the respective optical characteristic values represented by the respective electric signals produced by the image-take elements is associated with a position of a corresponding one of the picture elements in the physical image. The image data stored in the image-data memory are used to reproduce the physical image. In addition, the image data representing the physical image are actually present and accordingly the physical image can be said as an actual image in contrast to a virtual image described later. The screen on which the actual image is present can be said as the xe2x80x9cactualxe2x80x9d screen. Image data representing the actual image may be analog or digital data that represent respective magnitudes of electric signals produced by the image-take elements, for example, sets of multiple-level digital data (i.e., sets of half-tone data) each set of which can take one of 256 discrete values, or sets of binary data each set of which indicates whether the magnitude of a corresponding electric signal is greater than a threshold value. In the case where the image of the object represented by the image data is a discrete-type image defined by a matrix of picture elements having respective optical characteristic values, each pair of points or each search line are defined on a picture-element basis. Thus, the discrete-type image is suitable for those cases where the size of each picture element is sufficiently small or the degree of accuracy required is considerably low. Meanwhile, the xe2x80x9ccontinuous-typexe2x80x9d image whose arbitrary point has an optical characteristic value may be an image which is defined, providing that the respective electric signals produced by the image-take elements of the image-take surface (i.e., the physical screen) represent respective optical characteristic values of respective center points of the image-take elements, by a continuous curved surface which consists of a set of continuous optical characteristic values and which contains the respective optical characteristic values of the respective center points of the image-take elements. Data representing a mathematic formula defining the curved surface may be stored as the image data in the image-data memory. Thus, the continuous-type image is a sort of actual image. In contrast, without producing in advance the data representing the entire curved surface, it is possible to calculate, each time an arbitrary point is designated on a screen, an optical characteristic value corresponding to the designated point only. In the last case, no actual image is present but an image is just presumed to be present. Thus, this image is referred to as a xe2x80x9cvirtualxe2x80x9d image, and the screen on which the virtual image is present is referred to as a xe2x80x9cvirtualxe2x80x9d screen. Since the present object inspecting apparatus employs the virtual image and the virtual screen, it can quickly and accurately measure the object and inspect the same for finding one or more possible defects.
(12) According to a twelfth feature of the present invention that includes any one of the seventh to eleventh features (7) to (11), the measuring device comprises a seek-template data memory in which seek-template data representing a seek template is stored, the seek template comprising a plurality of pairs of points, the two points of each pair out of the pairs being distant from each other; object judging means for making, when the seek template represented by the seek-template data is superposed on a screen on which the image represented by the image data is present and respective optical characteristic values corresponding to the two points of the each pair satisfy a predetermined condition relating to an amount of difference between respective optical characteristic values inside and outside the edge of the image, an individual positive judgment that one of the two points of the each pair is located inside the edge of the image and the other point of the each pair is located outside the edge of the image, and further making, when the individual positive judgment is made with respect to the two points of each pair out of not less than a predetermined amount of the pairs, a total positive judgment that the object is an object being sought by the seek template; a measure-template data memory in which measure-template data representing a measure template is stored, the measure template comprising a plurality of measuring search lines; and edge-point-position determining means for superposing the measure template represented by the measure-template data, on the screen on which the image represented by the image data is present, and determining respective positions of respective edge points at which the measuring search lines intersect the edge of the image. The measuring device of the present object inspecting apparatus can quickly measure the position, rotation angle, and/or dimensions of an object. In addition, the present apparatus can produce inspecting search lines similar to the measuring search lines of the measure template, and use the inspecting search lines for inspecting the object. That is, the present apparatus can easily measure the position, rotation angle, and/or dimensions of an object which may not take constant position or rotation angle, and can subsequently inspect the object for finding its defect or defects. Hereinafter, a set of inspecting search lines will be referred as an inspect template.
(13) According to a thirteenth feature of the present invention that includes the twelfth feature (12), the object inspecting apparatus further comprises seek-template producing means for automatically producing the seek-template data based on reference-object defining data defining a reference object. In the case where the object takes substantially constant position and rotation angle, the present object inspecting apparatus may have only to produce a single batch of seek-template data (or a single seek template) based on the reference-object defining data. On the other hand, in the case where an object takes an inconstant position and/or an inconstant rotation angle, the present apparatus may need to first produce a master seek template corresponding to the reference object which takes a reference position and/or a reference rotation angle, and then produce at least one individual seek template by modifying the reference position and/or rotation angle of the reference object. Since the present apparatus can automatically produce the seek template, the operator is free from this task. Thus, the present object inspecting apparatus can be used easily and efficiently.
(14) According to a fourteenth feature of the present invention that includes the twelfth or thirteenth feature (12) or (13), the object inspecting apparatus further comprises measure-template producing means for automatically producing the measure-template data based on the seek-template data representing the seek template. In the case where an object takes substantially constant position and rotation angle, the present objecting inspecting apparatus can easily determine a single seek template suitable for seeking the object and accordingly can easily determine a single measure template suitable for measuring the object. On the other hand, in the case where an object takes an inconstant position and/or an inconstant rotation angle, the present apparatus needs to produce a plurality of seek templates and use each seek template for judging whether the object being image-processed is an object being sought. Thus, in the latter case, the present apparatus produces a measure template based on one seek template with which the object being image-processed is judged as an object being sought. Since the present apparatus can easily obtain an effective measure template, it can quickly measure the position, rotation angle, and/or dimensions of the object and can quickly inspect the same for finding one or more possible defects.
(15) According to a fifteenth feature of the present invention that includes the fourteenth feature (14), the object inspecting apparatus further comprises re-measure-template producing means for automatically producing a re-measure template as a new measure template for re-measurement of the object, based on at least one of a position, a rotation angle, and a dimension of the object measured using the measure template. The present object inspecting apparatus automatically produces a re-measure template based on the position, rotation angle, and/or dimension or dimensions of an object measured using one measure template. Therefore, the present apparatus can produce the re-measure template which has smaller errors with respect to the object than the one measure template. Thus, the present apparatus can more accurately measure the object with the re-measure template having the smaller errors.
(16) According to a sixteenth feature of the present invention, there is provided a recording medium on which an object-inspecting control program is recorded which is readable by a computer and is usable by the computer to inspect an object based on image data representing an image of the object taken by an image taking device, the image data being stored in an image-data memory, the program comprising the steps of producing at least one of (a) at least one negative search line which is expected not to intersect an edge of the image of the object represented by the image data, an optical characteristic of the image significantly changing at the edge, and (b) at least one positive search line which is expected to intersect the edge of the image, and judging that the object passes the inspection, when a condition is satisfied that when the at least one negative search line is produced, the at least one negative search line does not intersect the edge of the image of the object and/or, when the at least one positive search line is produced, the at least one positive search line intersects the edge of the image of the object, and judging that the object does not pass the inspection, when the condition is not satisfied.
(17) According to a seventeenth feature of the present invention, there is provided an object inspecting apparatus, comprising an image taking device which takes an image of an object; an image-data memory in which image data representing the taken image of the object is stored; a measuring device which measures, based on the image data stored in the image-data memory, at least one of a position, a rotation angle, and a dimension of the object; and judging means for judging whether the object has an unexpected edge while taking into account the measured one of the position, the rotation angle, and the dimension of the object, and thereby judging whether the object has a defect. The present object inspecting apparatus can inspect, with no problem, an object which may not take constant position, rotation angle, and/or dimension or dimensions. To this end, the present apparatus may employ either the above-described seek template, measure template, and inspect template, or other appropriate means.
(18) According to an eighteenth feature of the present invention that includes the seventeenth feature (17), the image taking device comprises a telecentric optical system.
(19) According to a nineteenth feature of the present invention that includes the eighteenth feature (18), the telecentric optical system comprises a reflecting plane surface which reflects light beams; a first lens system whose optical axis is perpendicular to the reflecting plane surface and which converges the light beams reflected by the plane surface, to a focal point of the first lens system; an object supporting device which supports the object at a position intermediate between the reflecting plane surface and the first lens system; a beam splitter which has a reflecting surface and which is provided on one of opposite sides of the first lens system that is more distant from the reflecting plane surface than the other side; an orifice which is provided at a position which is plane-symmetric with the focal point of the first lens system with respect to the reflecting surface of the beam splitter; a light source which is provided on one of opposite sides of the orifice that is more distant from the beam splitter than the other side; and a second lens system which has a focal point at the focal point of the first lens system and which converts the light beams which have been converged by the first lens system and have passed through the beam splitter, into parallel light beams, and the image taking device comprises an image-take sensor which is provided at a position where the image-take sensor takes an image formed by the parallel light beams provided by the second lens system.